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This symposium compares the^t^ird grade' oiitcoiiQe 
fprcifiles of Follov Through curricular models classified on the basis 
of their, theoretical undferpinni^pgs as (1) .behavior ally progjramed/ (2) 
cognitive-discovery/ or (3) <3peli cXassrpom. putcome p^rofiles ar^ 
presented for academic te^ts^ eonteptlial prablem-solving^ and 
sfelect;;e^ affective measures^ a^d c^urricular differences in parent >and 
teac^^r attitu.des are also departed. In^addition, the extent -^^o which 
similar models differ^ the pignif icanc^/of site-to-site variations^ 
rand the analytic iastie^involved in csomparing the various ' models are 
discussed. (JhB) . ^ [ ' \ < ' . ' 
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DESCRIPTION OF THE STUDY ' 

"Thomas Ferb ' 

, Abt Associates Inc. 
. ' 55 •Wheeler Street 

Cambridge, Wass. 02138 * ' , 4 

In the presentations which follow, three curricular approaches, one 
Cognitive, orte Behavioral, one Open are presented. -^Each of the ap- 
proaches is' represented by two Follow Through programs;^ however, the parti- 
cular Follow Through, programs apd the sites involved are not identified by 
name. The letters A through F are used for the identification of programs^ 
and numbers for sites. The programs are. not identified because .the focus 
of this symposium is .on the individual curriculum and its particular impAct, 
Furthermore, the results presented are based on the Cohort II stream of 
Foliov Through data which is not the primary focus for impact evaluation, ' 
The Cohort III stream .which is" currently! being analyzed is the main focus 
for the evaluation purposes and will be reported on lajner ,this yelr. 

lt\ the following presentations, a particular f onSat ^s followed. 
The preset! tations begin with a definition bf the curriculaf approach, fol- 
loved by a specification o£ the sample in terms of the particular children 
and the sites involved, followed by some data on the reported attitudes and 
practices of teachers, and finally an explication of the impact^ of the pro- 
grams on children is present;^ed in terms of norm scores obtained at the end 
of third grade and in terms of gains or progress from the beginning/ of^lcTn- 
dergarten to the end of third^^grade. 

In this presentation I 'will deal with some definitional and inter- 
pretive detail concerning' the, sample characteristics ^ the teacher measures 
utilized, th^ child outctane measures, the analysis "^strate^y, and a majpr 
weakness of the study related to Head Start and presihool. Let us begin 
witji the sample. The children utilized for this stu<iy participated either 
ip ithe FolloV Thrpugh program or one of the selected comparis^on groups in ^ 
' the Coho^rt II stream of the Follow Through evaluation. This, groug, began . 
kindergarten in 1970 and completed third grade in^l^?'^". .In general, the 
Follow Through "children are .substantially .below the national population, on > 
many socioeconomic status characteristics. The. median income of the 'national 



population in 1970 was $9500, while family median income in Follow Throug-h 
was only $4400. This -large' income difference between the, Follow Through 
children and the national population suggests that the Follow Through pro- 
graiTwas successful in recruiting a poverty level sample of "pupils, other 
characteristics of the sample substantiate this general disadvantage. 
'Eighty-poe percent of the ?T children participated j.n one or atiother pre-' 
school, whereas only nine percent of kindergarten children in:the national 
population in 1970 participated in preschool. The majority of 'children in 
Follow Through (75%^ are Black whereas only 11% of the national'population 
IS Black. Faraiiy size in Follow Through averages six whereas the- national 
population in 1-970' averaged only 3.5 Thus the' Follow Through grdufi appears 
to be substantially different frcffl the community at large. j 

The Follow Through and comparison groups are not nearly so mis- 
matched, rne characteristics of the overall Follow Through and comparison 
group are quite similar. The median income for the Follow Throu5jVIgib\3p, 
again,, xs $4400, whereas' the median income for the comparison group is- ' 
$5400. Seventy.-five percent of the FT group is Black hvA 66% of the com- 
parison group, i's Black. A major difference between the groups does- appear 
o^a prescfidoi/ , Sighty-ohe perc^^'of the Follow through group graduatetiV-— 
from^^aa: start or other preschool; whe'r^as' only 57% of the compa^l^^group 

preschool graduates , •ihese^ differences suggest that the Follow Through 
g'roup is somewhat disadvantaged to the comparison group. 

It must/be-ppinted out that t^e' diskdvantage of the FT group rela- 
tive ti the comparison group is highl^' ;^^ariable across sites. In some 
locales the Follow Through group and comi>a:;ison group are very gimilar 
whfereas in others the comparison .gxoup has., a substantial socioetphomic ad- 
vantage. The degree to Vhich' the Follow Through and Qomparison group are* 
comparaiDle has some obvious implications in .t^t different groups present 
different problems, not only in terms of ' the social' and, educational 'probe Ims^ 
with which the programs must deal but also in terms of analysis, problems ' 
related to ^estimating the progress that the children h^ve achieved over .the 
foui^-year period of participation in Follpw Through.'; ' Furthermore, the 
likelihood that the comparison children participated in some other ^mpensa-,. 
tory program probably varies among sites. This lack of compa'rabi^^^^ yill * 
be Considered be^-ow when the analytic strategy is described. / • 



* - Let xis now turn to the teacher measures bhat are utilized 

teacher data reported here deal exclus^iv^Iy with third gradj6 teac^^ ques- 
tionnaires, again from the C^ iiort LI gr6up_._ We, make use ,<if fi;^ dimensions 
or clusters which attempt to rfnkifaG^rize the i parrii nc^/pny^^^t^mpr^^ 
children and may serve as proxy ..variables for^.:QXQ^X^ impl 
clusters include: , 1) teacb 

. the classroom? is cc 




bility, that is, the degree'*^€c5'''wfTich the childre 
exercise freedom of choice; 3) individualizati 
degree of emphasis on small group instruction; 4) ara(^f>mT_q 



emphasis, that is, the degree. to which teachers emphasize basic skills ^ath^ 
than socio-emotional developmeAt , and, finally, 5) teacher /sati^sf action with 
the sponsor's program, that is, the degrees of the'^Follow Through teacher's 
liking' fo'r ^^nd. assessment of the usefulness of the sponsoi-'^ program. These 
dimensions were developed fr'om the Pearso*!^'^rrelation Matrix of items from 
the teacher questionnaire. The correlations were submitfed to a cluster 
. analysis using ap average linkage betv/een merged groups to identify the di- 
mensit>ns or clusters. The methods of reciprocal averages was then employed 
to develop item weights for eabh item response in each of the cluster vari- 
ables. The intiernal consistency of the scales was assessed and they were 
found to have adequate reliability. Finally, the cluster scores were stan- 
dardized, across the Fol low-trough and comparison group teachers "using a 
mean equal to 50 and a standard deviation equal to' 10. Noire, that these data 
are being used to get a sense of program implementation within a site. The 
different curricular approaches. Behavioral, Open, and Cognitive, have very 
different objectives with regard to the learning environment and hopefully 
these five dimensions character i"%e the goals cind differences. 

Please note that Figure 1 shows a T score for each of these teacher 
dimensions — teacher centeredness , classroom flexibility/ individual iZfition 
of ixistmc^ion, emotional vs academic emphasis, and teacher satisfaction 
within the program; a value is shown for each of the sites in which a par- 



ticular program is pperating. The sites^ are \dif f erentiated by the key at 
the top of the Figure. Since the scores have bean standardized, differences 
* between the value for a particular site and the overall average can be di- 
, rectly interpreted from the table in terms gf T Icore differences'. Our dife- 
cussion is restricted to differences that are equal to or greater thart-one- ' 
• ' r. 
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quarter of a standard deviation , that is, 2,5 T score units. .All of the 
scales are ordered so tha.t higher scores indicate a higher degree of the 
dimension. Thus, a site with a value on teacher centeredpess of ^0 may be 
more structured i that a site with an average val-ue of 50/^ Th^g- tentative na- 
ire of these data .must be emphasized, First'of all, the data are fcrased 
ly/'on third graS^-'-t^A^^ers . Secondly,, within sitK^the number of teachers 
Jjg;;^-3o^et.imes7j5uij;e low. r^4r€hennore , since the da<:a bas 

Cher 'repdi^Sivand not on independent observation' their "Sf^e" for either 
haracter-izing the ^^ai?ni ng environment of children or as a "^oxy variable ^ 
the implementatioa,(5^*5'"pa;;'Hcin3S~^ may- be limited. 

as now rurn to the chilBoutcome lijieasures. ^attery^?vOf' existing 
d tests was chosen amd modified for 'the siationai 'evaluation out- 
•me^eS^U?fes . Af zer^careful consideration, this ba^ctery was '|el*ebt^kas a 
/best compromTs^^ between .the needs for acOountability and, the dif f iculty^f ^ 
/pleasuring sponsors* diverse goals, an^ objectives.^ Th^se measures allow 
comparisons to be n^ade between the...^Foll^ Through and comparison groups on 
the attainment of basic slciIl^S"," ^ cognitive and conceptual skills, and af-'' . 
fective measures i^cludiog both self -esteenr^^^k^d attitude toward school and 
learning. The measures include .the Mevropolitan ^S^ievement Test with its * 
SpeTllng^r "Reading, and Mathematics standard subtests; the Raven's Coloured ^ 
progressive. Matrices, which measure a .child's problem-solving aijil^ty ii^'"^^ 

visual and perceptual tasks; and the Coopersirrith Self *-£3j:eem Inventory, a non- 
' cognitive measure designed to assess the^'extent to which children taJce 
resoonsibillty for themselves or attribute th^ir successes orf failure^ to 
the operation of .internal or external forces- ' - 

Let us now -turn to the cassessmant of the impacts of the curricular 
programs. Here we are interested in two questions, 1) "How does the pro- 
gress of Follow Through children comjJare with that of comparison children?" 
and, 2) "How does the MAT performance of the Follow Through children com- 
pare wi A that of^ the national' norms group?" Since the Follow Through and 
comparison children are non-equivalent groups, that is, since the groups 
diffei: on a variety of background characte);:istics including average achieve- 
ment level at entry to kindergarten, it is neqessary to look at the .progress 
that the children have made ;during their particiipation in the program rather 
than compare the raw scor^es of the treatment and co^piarison groups. In 
order to accomplis,h this estimatipn of the, progress and answer, the question. 



"How does the progress of the Foll^ Through child comp^e with tH'at df the 
comparison child?" regression mod e\ was developed^ for each outcotae, which 
y comoarison children, "'fee. regression: 

It 



utilized only comoarison children. Itie, regressionymodel involved the pre-^ 
tes^^ score on the Wide Range Achievement Test at entry to kindergarten, 
, family income, occupation, ^thnic memberships^ s^, age at entry to 'kinder-* 
gart^n, first j^ang^tage the child, and a variety of other interactio/is. 
Since the model utilized only comparisoil children, it essenti^illy descr^ibes 
progress for that groi:^r that is, given certain background' characteristics 
and a particular WRAT score at entry 'tp "S&cSSpX^, the model yields a prej^icted 
posttest si6re for each- child. , Th^' coefficients 'that m^e-ufi'the model ^ ' 
indicate the contribution of. each background f act^or and the p^fces^t^p E^tr 
test .level for the average comparison child. This' model that is the ^ ^e t^ b;|r* 
coefficients was then applied to the Follow Through group.,: iSie 'application' 
of the model to .the FT group yieldedva predicted score for each*child ^ 
that is, the score that the FOllow^JJiro^h child would have received if he; ^ 
had participated ill the comparison'"grou^ experience. .T^e impact of the 
^rtentis^ assessed by explorin^r-^e^iscrepcmcy iDetweeir^^e^,,fireSi^^ 
\score ai)d^SS^|rct^al score that the child obtained^ She subtraction^yields. ) 
an effect:" a posi^ve effect if the program has had a f avorafatia Jjtipact-> and 
""tKe"* child's score rs higher than expected; , a zero effect, that is^, an effect 
that is not statistij^^Jly different fjrom zero if the Follow Through child 
has achieved or pfogi;ess'ed as much in the^ Follow Through experience as he 
would have been expected to achieve in the non-J'ollow .Through ^perience^?- ^ 
and' a negative effect if the child hajs ah> observed score which is.l^^'& than 
'his predicted score, suggesting tH^t the child might have performed .ifetter . 

had he been in the comparison classroom rather ^than in the Eolloy Through 

, 1) " 

classroom. Triese effects are shown in Table 1 for ea^ch of the" outcome , " 
' . ^ . . . ' . i! • / " ' ^ . ' " 

areas. In the table, differences beween Follow Through and non-Follow ^ 

Through are expressed in standard deviation un^-'ta. Thus a difference of ' 

-.44 indicates nearly one-half a standard deviation difference favoring,, 



nonrFollowt Through. We restrict our 
statistically different f^rom zero and 



iiscuS'Sion to effects which are both 
which are also as large as^'on^- 



quart^r of a standard deviation which lis basically an'arbitrary criterion 

a device to understand the educational , significance , of a.^diff erence 
between groups. That is, some differences which may be statistically sig- 
nificant are not in fact of any educational consequence, and one-quarter 



of a'standard deviation difference provides at least a face valid criterion 
for 'educational significance. , , \' '~ 

The coroparlson^ of, the 5*bllow Through groups within a particular site 
with this overall comparison piodel has several advantages. First,, the re- 
^gression coefficients, .developed using only the comparison children, are 
not affected by the treatment. , If a Follow_ Through ^program produced greater 
or smaller gains in some subgroup of "the children with which the prbgiram 
was dealing, ,for example, if children ^th higher achievement scores in . 
Follow 'through gained more, and if the coef^f^ient were develo^ei^ as for ,an 
^alysis of covariance using all children, the coefficient representing^ the 
contribution of a background measure to the outcome measttrLe would be biased 



by treatment. A second advantage of the use of t^e overall comparison model 
is th^.t it provides 'a common 'comparison for all program^ and for all slrtes 

The second "method? of assessing the impact of the program is found 
in the assessment of the norm scores of the children at the end qf third 
grade: 'Although it is difficult or impossible .to attribute to a. program 
the noinnative status, of a group at t^e' end of third grade, .it is certainly 
^not expec£^d that' the^e groups would be performing at or near grade level, 
since the majority of children are "disadvantaged.? fSixthermore , the legal 
definition of educational disadvantage includes a, criterion concerning 'the 
performance pf children' in terms of grade level** The legal definition 'of 
disadvantage defines students who, :are performing one ojc morfe years behind 
the expect^ed grad^ level as educationally disa4vantaged . This then provides 
'a natural <s;r iter ion for exploring the status of groups at the. end of third 
grade ^ Groups where the median score is at or near grade J.evel cannot be 
construed a,s disadvant'aiged, at least not legally, and therefore programs or 
^sites in which t^he^ ytedian score is close to grade level can be construed as.' 
successful/ The grade leve^» performance of" the children "in each of. thfe^ , 
sites associated, with the program ar*e .shown in charts such as Figure 3. 
The figu^'es show t^e median grade level for \AkX Reading, Math,^ and Spelling ^ 
for feach site and the percen'tile -of the median, ' / * v ' 

' ; You may as^c^ '"Why are theSe people proposing the use of grade 
equivalent scores which have h^n in disrepute for a long time in the fi^eld 



* Section 17a of . General Provisions Act as fended by public, l^w. 

93-308 define? educajiibnal disadvantage as achievement which^is one 6r-more, 
ears behind that expected at the appropriate grade level'.* 
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of educational evaluation?" Th^.s is ind^eS legitimate questiori/; 




opoortunity to attribute the level of p^fformjcmce of the children within a ^\ 

" , ^ ^ ' 9 \ . ' { ^ *' . • ^ ^ 

■given group is minimalVor absent wh4ri only grade equivalent 'scores at the- ^ 

iT"" * * . ' ^ ' % u r p ^ ^ , ' , ' 

end M third grade ^are available. Stiil', tjiese fcores^ hay,e tace, validity • 

in terms of their use in' schools:' Teacher^s^ ngt ,all, but some, utilize ti^nu 
scores t9 make judgments about ,the assignment of children to special pro- 
grams. Furthermore, norm scores are often used to communicate a ch-ilcf^'s 
^status art^'progress to parents. The ^tatement that V^ommyjis reading a 
grade lev'el ahead' or behind" has great signifio%nee for Tommy, 4iis parents, 
and his teacher. Furthermore, we probably all agree that the preferred 
estimate of program impact, that is,' the. regression es.tJiflaate,. TOich is pro- 
vided by comparing the observed and .predicted~""s??ares of the children at the 
end of third, grade Js also suspect and subject to^ a variety^ of criticisms. 
The ma^iority of these criticisms suggest that regression approaches Under* 
estimate '^Isreatment e-ffects. That is, they show no, effect on occasion, or 



neg^ativ^^ff ecbSssWhen-. in fact the program has had a favorable impact. 'It is. 

. beyond the scope of tlvi^ presentation to go "into these problems in detail;; ^'j 
however', it would appear that both^^^rade equivalents and regression ^egti- *' 
mates ©^progress hav,e a. variety^ c^f^ distin^ct advantages an^disadvantages . 
It makes^-^ense to capitalize oo the ln|9rmation^ available an^ combine, ^ -v 

•these two information ^sources about programs and the status of ^children aiid^'"^ 
define a favorable p^gxari. impact in a somewhit unorthrodox va^ 'Basically, 
a favorable program 'impact can be defined either as ^reai$r prbgifes^ in ^ 
Follow Through than expected^^or as progress in PqIIow Througfi* which is no 
less than expected, paired with perf onaance .on the horm^ which are^at or 

^ near grade level. This definition of program in^ct'.is utilized in^the 
'following pne^eni^arttons. • ^ . . . ^ ' ^-i' 



Let us now turAjto some of the limitations ^of* this ^stu^ an^i^o&^the; ^ 
data to be pteseirfeed^in t^is ^^ymposium. Indeed, we '^Jcou Id ^devote the entires- 
symposium to an explora^on.of the limitations of the**Foilov/ Through evalHia- . 
" tion.^" There \re various ^design .constraints, aspects* of ^he ^measurement, 
battery, implemjentatitSn issuer, attrition issues,' analysis '^issue^/ ^and , 
others which we could consider in detail. However, one nvajOr litnitatiorf ^ , - 
.seems to stand out^among the problems This is ^the issue of He^ad 5^^4t "4r^ ' 
preschool, and the possible consequence of n<5t utilizing this aspect 'df , ^ - 
children's background ia the model of scores. , First of all, I ^must point: 



patt^at this' aspect th^ cl;iiiaten^ s background was \ot selectively ^ig-' 
•nWed/ The^^riable was not uliilizfed ' in part b'ec^usfe'it: is severely con- . 
founded with site .and Follow Through. In sdSne sites' ^5% dr\the folic 



Low 



lildren may have preschool or Head--^Start experrence %s bnly Op% of 




the cdin^rs^l^^^ doe^ not make 'for a useful covafi- 

abl^: Furthermote^:^!e, mis>ii^ rat^ that is) information^ as ' 

^^^^^^ ^ • ' ^ ' ^- ^ 

to whether a child^ p^rt^^te^^t^^ ~ is intolerably hi^h in maW' 



cases. Since Follow Through "^^^int^^ed^^^ follow 'through on Head's tart, 
the absence of this variable is unfc5lrtunkte. -Furtheniiore, sihce preschool c 

' - ^ ' >^ . V 

Inay affect the pretest score, the Wi^e Range Achi,eveinent 'test / at entry sto 

nde^g^arten, may further bias against, t:Ke finding of treatment effects- since 
thfixe^genef^liy appears, to be more preschool graduates in Follow T^^ro^h^. 




a;- 



■ /! 

■ /' 




•'V- ■';>■■ ■ . / ... 




COGNITIVE -DISppVERy OJRRICU.IAR MODELS' IN PROJECT FpLJ-OW ' THROUGH 

\ ' , , John C.Lair^n, Ph.D. < - , 

* . Abt Associat^es" lnc« ^ "^^ { ' , ^ ^ 

\, . ' '-"^ 55 Wheeler Street . ' / / ' ' , ' " ' 

; • . ^ \ Cambridge,' Mass,. 02138 ■ ^ .v. . \/ 

Overview of thfe' Cognifeive'^Developpiental Mcx^eljS* c • * 

'/ TWo\of the major curricular toodels' in Project Follow Through exem- 

plify a'form^ of the coghitive-discQvery approach to early^ education/ 'They 
seek to .stimulate the child's basic cognitive abilities and problem-solving 
skills a5 th^ foundation for later academic achievement. In addition, they 
\ attempt to enhance the x:hild's geli-esteem and feelings of competence in ^ 
support of the child's curiosity, self -initiated exploration of thfe learning 
environment,- and self.-monitoring strategies for solving problems. The 
sponsorg'pf both these models see active parent involvement in the child's 
V learning at home and^ in ctjciss as'an essential component in the child's 

adjustment to the 'school and td-^leaming in general. . , 

. 'While botl^' approaches focus on ;the child's development of problem- 

* * . t 
solving, skills. Sponsor A emphasizes learning experiences structured around 

, a theoretical framework of ^ ci^ild. development in classification, seriation, 

* ^ ^ , ...» ;: 
and spatial and tempbraJ, relations, while , Sponsor B emphasizes a more open^ 

responsiveness' X9 a ra^ge 9^ child-directed, learnini^ interests. Both 

' sponsgrs stimulate self-initiated and self-mbnitoired learning activities 

in chilMreh,, but Spor>sor A stresses the child's prior 'planning and follow- 

up more than.dpes €ponstDr 3. Sponsor '^B sufSports self-confidence by mor.e 

consciously stressing mutual' respect ^f6r subcultural differences and ^ ' 

responding to ishe^ child ' s^'owi) i^idividuaL range of interests. Sponsor A - 

supports self-esteem by more directly encouraging self -expression df \ ^ ^ , 

thoughts, plans and, learning 'experiences in the written language.; The 

approadh iOf these splonsor.s' differs from the 'Behavioral approach In their 

different concepts of child development in their lack of specific 

emphasis on academic skills. WhUe their objectives are similar in many 

re'sDects to the^, Psychodynamic apprdach, their ' relAtiveiy^ greater emphasis 

on problem-solving, skills over sbcio^-^otional and interpersonal skills. 



diffferentiates^-' them the Matter. 
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^ Sample * ^ • . ^ 

We /Sites for the an'alysis of Spojisdr A , include two large^ cities 
and a southern coTiuTJunity* ^ The children' in* thes,e ^three sites come from some 
of the most economically disadvantaged faihilies in the total Bollow Through 
sample, 'With a mean'^nual inco of $3703, these sites reported some of 
the lowest income levels and highest pr9poi:tions' of single-parent families ♦ 
Their kindergarten entry scores w^'re .among th^lQwest in the total '^rvalytic 
sampi^. Fewer thati .*half of the mothers' had cc^njsleted high school/ 

' ^ The sites include^d t^ri' the^aly^sis of SponsorBvtjad generally higher 
levels of annual \inco!ne. t^itK ^ mean of '$5756) than thdsefbri^d in most 
other Follow Thrpagh ^ites* . A greater proposition of moth^er^' at the^ sites 
had compl"^ited highA school llhan was true for the Follow Through average, and 
the children'^coreAhigher tsnS^ademic tests at kinderg^ten entry "'than the 
mean of. the overall Analytic sample ♦ Thus, the environmental circumstances 
in Vhich this sponsor ^Jriplemented the. program were not so educationally 
handicapping *a5,tho$^ of 'Spori^>ii^ A. . \ 'i • 

il'eacher Rkport's-as Indicators » of Proqram> Imple mentation 

' ^""^ ■ t 1 . " , * 

. ' T^^-data in Eigures 1 and 2 .illu strati the extent to which the 

teachers in 'the "separate sites of these., programs eharacteirized their class- 

*};poms as: 1) Child-centfered v£ teacher-centered; 2) Flekibly structured 

*vs- iiif lexibly structured; 3) Individualized ^ group oriented; 4) Socio- 

^otion^L vs academically oriented; , and 5)^ Teacher's satisfaction with the 

" . * / , ' i i * * 

model: , • ^ * ' .; , 

^ * ' These dat^ .suggest that the cognitiv^e-deyelopmentdl models ^enerklly 
succeeded in iiqp dementing a child-centered, f lexib^^e^^^;^^ 

curriculum with^ a relatively greater emphasis on socio-^(3tional than aca- 
demic objectives. '.Although Sponsor A has only three sitts^ indl^ded' in the 
analysis, its' classrooms "scored consistently in the same dir^ection on all 
of the teacher reports. This uniformity of teacher reports, found in no 
other sponsor discussed pin th^s symposium, suggests that Sponsor A hacr 
we:^,l-integrated trainin<^ program with respect to the dimensions reported 
h^re. For Sponsor 3 the sites, were unanimously more child;^centered'^ and 
individualized than the average of all teachers In the analytic . Sample . 
There was, howeveK^ evidence of site variability in the Emotional-Academic 
Teacher Satisfaction, and Flexibility dimensions* ' Gi%en the complexity of 

i, - ■ } " ' ■■ 

10 • / 



' ' Figure 1 k ' ' ' ^ ^ 

liiaications of Implementation From Teacher QuesfionnaiVe Ckisters 

for^Sponsor A > - ^ ^. 

, Cognitive • Developmentai^Approach^ ^ 
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th^ Follow Through modgls^w we cann^ assume tiiat all meaningful curricul^r 
differences are registered on j^ust^^jkhese dimensions, nor can we assume 
that teacher reports directly ^orrespohd to the levels of iir5>lementa;tion 
desired ol: specified by sjJbnsQ^s, ^Th?sg data do indicate y however,, ; 
th'^t characteristics^ important to a cogiiiti^^T-deve^lpp^ approach 
were successf\illy commionicated ta the\ minds .ajxd understanding of 
teachers. ^ • ^ • 




Interpretation a& Curricular^^^pact on Child Outccmes' ; \^ 

Considering nexp^'^^ child outcccaes at ^the end qf third 
examine cxorricular^f^cts from three set^s/of data: first, the. comparison 
of the Follow TJarough groxips to the ^>poled noh-Follpw Through- grov^); 
second, the cootfSfet of jthe, Follow iJirough^/grougs to jiational achievemtot 
norms; and finally the qualitative contrast of curricxilar effects acrctss 
^ diffe^lit^ domains.^ . , * , ^ ' 

Levels of Curricular Impact ^ 

The data in Figures -3 "and 4 suggest how the performance levels of 
Low Through groups differ from ti>o3e expected of i^imilar groups in 
non-FoMow Through.^ In these figures the bars extending to the right 

\ vV.*' J i ' 

of the csti^par suggest that the Foliow Through Site performed at a level 

higher than^^*6\pre^tete^^^^ regression model, while bars extending 

to the left suggest tfieV converse. T In the academic areas, the effects,. 

suggest that the OTildreh in Sponspr A performed somewhat ibwer tJian 

would be expected. Of the tnree ^ites', the one which indicated higher " 

than 'expected "levels of performance on Spelling, self-esteem/ - and 

' ~> ' * • ' . . ♦ 

conceptual* problem sblving relative to the other "two sites, is that 

site, in* which -the teachers also reported higher levels .of satisfaction, 

greater classroom' flexibility/ and the least teache'r-centere^ness • The 

ciassrbpm emphasis also tended/^^jidte strongly * toward the socio-emotional 

rather than the academic direction*^ 

The^ academic effects for Sponsor B are somewhat more^taixed aqrpss 

sites than for .Sponsor A. T^ro of the seven sites indicate small " but *£:on'^ 

si^tently better than predicted performance on ^ath and Reading, while^fou'r 

ofr the seven sites indicate per^formance levels slightly to markedly lower 

• • . 13 • " 
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Sponsor Effects on Seven Third Grade 
Tests, Expressed in Standard Deviation ' 
Units of Raw Score, For Sponsor A 
Cognitive-Development Approach 
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Figure 4 

Sponsor Effects on Seven Third Grade 
Tests, Expressed in Standard Deviation 
Units of Raw Score, For Sponsor ^ 
Cognitive-Development Approach 
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than pred4.ctec3 by the ^regression analysis. All sites scored lower than ex- • 
pected in Spelling. As was the case for Sponsor A^ the teachers in the sites 
with relatively higher levels of effect tended tfb report the highest le^ls 
of satisfaction and relatively high levels of socio-emotional emphasis,- - 
non-teacher-cente.redness and flexibility in. the classroom organization. 

It is difficult. with these data to draw too close a correspondence " . 
between levels of teacher reports and levels- of curricular effects, because 
of both imprecision of the measures and lack of consistency between these 
domains on the low end of the scores. However, within \he tange -of site 
variability for both sponsors, the, most positive academic effects seem to 
follow those sites wherein the dynamics of the .cognitive-developmental 
approach are most in evidenc^. > , ' * ' . • • 

In the affective domain and conceptual problem-solving test, the 
ef Sects bf Sponsor A generally parallel those. observed in the academic 
domain. Two of the- three site^ tended to perform at lover levels than would 
be expected, while the third site tended to perform at higher leVels than 
predicted by t^e regression equations, in two sites there is some indica- 
tion that where lower academic performance is found, affective spores ar^ 
not so low, or where higher than expected academic scores are 'fcHind, the 
, affective scores are still higher. For Sponsor B ^the affective land problem- 
^solving areas indicate a pattern of sit^ variability similar to i that found 
in the academic; ai'eas. ,As noted above, in sites where children score lower than 
expected in academics, they* seem to perform relatively better oii the affective tests 

To^ sumnTiarize the comparison of the cognitive-developmental curricula 
to -the pooled non-Follow Through ^ the clearest generalization s^ems to be 

site variability • This analysis suggests that while~ sme sitesi show higher 

' ' ^ ' . • ---- 1 . 

than expected performahce in^ severatl «areas, most of the sites for tljis 

appro^hjtend to show performance levels somewhat lower than 'wotild be* ex- 
pected' from* 'the -regression analysis. There is some evidence to suggest- 
that re^§tively higher clfevels of perform^ce appear in those sites report^ 
ing a higher degree of classrociu flexibility, child-ceriteredness and teacher 
satisfaction. - ^ 

While the' results of the regression analysis provide comparisons of « 
the^ program^ to" th^ pooled non-Foliow Through group, grade equivalent and 
st;andardi2ed scp^res on* the Metropolitan Achievement Tests^ in Figures 5 and 
6 prov^de^a comparison pf the mo'dels to another external, criterion, the ,, 



Figure 5 

Grade Level Equivalent and Standardized Scores on MAT Subtests 
(Reading, Math, and Spelling) with Income Levels for 
COHORT II Sites in Sponsor A 
Cognitive — Developmental Approach ^ 
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Figures ' 
* Grade Level Equivalents and Standardized Scores on MAT Subtests 
(Reading, Math, and Spelling) with Income Levels for 
COHORT II Sit-es in Sponsors _ ^ 
Cognitive - Developmental Approach 
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:ita±i^n5p^;i5^Ta^fi:^;^;^^ The. comparisons ' just discussed suggest 

o^ progfaih impact^.x^/hiie" thei^grade equivalent-^scores suggest the 



Status pt the Follow .Ti^tough sites relati^;^ ^PJ^ country at- large by the 
end of the child'ren/ s tenure in the program, for Sponsqr A the academic 
'stattis^of the tnree sites can be rank-ordered- as : I) within ^rade-level 
performance, 1.^) bdrctear-ione gracje level perf ox:m«Lnce/ and 3) -belAw grade 
3.evei performance. This order dWs not correspond directj.y with th^ ^P^onsor 
effect's analysis/ since the Ibigher^^^ha^-expected effects- were found in tTie 



borderline ^gr/kde level site* The, data suggest^ .that the order of grade 



level performance does correspc^ndV however, with t 



of the sites ' 




mean income ievelfe* ,^ 

For /Sponsor B we find a similar relationship between grade rev^L 
performance rand site me^n income. Four of the sites are within ^the graS^-^ 
level performance criterion, -and thr^ee sites- are bellow grade-level per form- 
jance eriteridn. What is the relationship of the gtade equivalence scofes 
/to the program effects discussed above? One hypothesis' would suggest. , that 
prbgram effects are more related to levels of implementation,/ whxle- grade 
*equiv*alen1t status is related more to income level. As^ a corplXary, it may. 
be that si,tes wiih' g;:a^e equivalence -levels , higher than that expecte4 frocp 
' their income level? obtained these *acijievement levels in part through' a 
positive impact from the .sponsor' s progr-am. ss - ... 

Qualitative Xssessment''of CUifr-iculajr Impact - - % ' • ^, . / ^ 
Anai^^isip^the^levels^ of^impact^ f or^^th^^^rricular model seeJcs*. to ^ 
answer the ^mestion^ !^How mudff impact did a cuj^ricuium .haye on chjLld out- 
comes:?" Ah appropriate answer to thjis d^aestXon is best^ obtained through a 
' |figprqus/ex5§riitiOTtaL[^ yet the metal, ^6^^ our experimental logic in- 

evitably Jd^es much of it^^^temper when, subjected to ..the pressures And 
strains -Of ^ the r^al-!^world ehvironment teachers and childreh scattered ' 
across the entire country.-. In a quasi-ekperimental. setting,^ the results* 
^r-oC^our ^iialy^es agre sensitive to a" numbbi: of circumstances such^.as the 

desree^f '^iMlarltV^^be^ ffie*^Hpw Through and -^oinparison^ groups, the 
S -^absolute^ l.eyel' ol^|J<^^;t^ty -w^^^ the^Jocal^^oommunity gttrde^^^ and 
^in 'which the-sponsair. seek^^^^in^^^h^j^ - 
instructional ^program the^es,t^i^n^^biiQ^:s^i^^L^^ 
f ^atur^s of'^ the. community . 
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' A fuller corisideration t3f ' curricular impact, however, (demands con-^ 
, slderation of still ^ another set o£ factors/ Within the constraints of any 

one 'site, wt^tVas the quality of the' program *s impact? Considering the 
; , coittent areas relative to each other, in what domains were tjie program's 
effect^ mqst^^sitive (or' least negative) compared to those areas vfhich 
^nded to be the l^st, to respohd to the^ curri^cular influences? The factors 
affe^rting^ ,issues^^r^fer"~J^ dyna^jics of the learning situation, or 
the fprm of educational ^perience mold^ by the program. To what extent 
are learning' activitx^^ self-pifected' or te^^cher controlled? What is the 
scope of ^t^neralization^^krvd fjlexi'bility. of application tlo whilh the learn- ' 
ing' i^ applied? Does , the learner acquire self-esteem^""bJ:^ugh taking, r isles 
and discovering competencies J or through following directions well, or by ' 



learning^ to eh jfoy* learning itself and mutual social support? Th^se issues, 
too,- mast ^be, examined .in as^esging a curricular ptogram's influence.' 
• : « ,As a' heuristic appr:oach. to this question we have rank-ordered ''.the ^ 

'/'ieyel^ 'of effect across the seven outcome -measures within each site. That^*" 
outcome' which shov^ed^^tbel highest positive (ot least negative) 'effect, 
regardless of ^.ts level, was given the highest rank; and-vice versa. We- ' . ' 
then determined the mean rank for^'^e^ch cont:ent area across^the sites .of 'the 
cognitive-developmental - apjproach* , T^e^ results , ^indicated '^in Table' 1, su^gelt 
that for the cognitive-discovery appirq^ch, relatively nw^re ' posi'tive/efiects 
. were pbtained in the , area o£ self -esteeig^ ■ foMt>W^d next by ^ internal /loc|^^ / 
of control and prcrfDlemrfeolvi^ng . ' Conver^tflY/ Math and Spelling'^w^re thi^.^ast 
areas to show curricular effects. .Thev,preem£nence of "thefaffectivel areas * •i^ ^ 
"over, tjie acakemit areas „ corresponds ^ to *the relatively gr^a'ter teacb^ei empha-v 
^ sis reported QH' i:he^ factors bf child -center^H, s©cio-i(todtiona,l focus , ^ and^ * 
>i^;.^a^s^Qo?til^^^ this' curticular 'model. . gased on fhe ,gbaX^ of the 

'.;approabhv hpwfeverJ^fe-^S^uld have e:(peete<j" tc? iee 'the ^J^rorfLem-.^olving* area 
\ r emerge . a$^ that whoxh is fitst> af f ^bted 'poddtiveiy ty the curiiquluffil These / 
data sug'gest that the ^dynodibs of the '^ognd.tiye--developn^fenta:j. cur3Acul\am . 
-correspond in part with the- aspirati-dns of the model, ' Further %studies ' , " 
designed'-' speoificaflly to examine \svy:h dynamics ^ and 'to d^termi^e the rela- ^ 
% tionship Of ' such le^min^ orieritations, to: future School Recess must be' 
conducted in order td gain.'^ fuller understanding: pf th^'ultimate utility,^ 
of th^'^cogni^tiye-developmental curriculum in prepiiring -^the cftiildjfor later^ 
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Introduction 



Follow , Through as ah experimental program was* implemented using ^ 
a strategy known as "planned variation." This strategy systematically 
int?:oduced a variety of well-defined programs into the kindergarten 
through t)^ird grade years of public^ education in certain school districts 
throughout the iJnited States, To operational ize thfe concept; of "planned 
^variation/' USOE involved educational specialists, each sponsoring a 
different educational approach in a variety of^^sj:6ool districts. The 
Follow Through progi^^ as it was/de|ined by' these matched^pairs of 

models and school districts consisted* of two sets of,.-<iOT^p6nents: social 

' ' X . ' ■ * ' ' . ' . ' 

service ana education. One s^t of canponents — sOcial services, parent, 

and coiranunity participation . — was itlandated in the 'original enabling 

Congre'ssional legislation. The second component/ education, was .the 

essence of plannei variation. ,Thi^ component , was comprised of fotir 

major Ntypes of ed\;cational strategies: ' • 

• Projects ^phajsi^ing a'^behavioraXly ^st^cture4 approach 
to acquire "Academic skills, 'fi^rticularly reading and 

, ~ arithmetic, • ' i ' f , . ^ V 

Projects stressing cognitive. thinJcing .thro^gh^ asking ^ \^ 

. and answering questions, problem solving, ancj^ creative 
writing, . I / ^ * , . ^ / . . ' 

• .Projects emphasizing Social .emotional development .and. 
encouraging .exploration and* discovery. in.,SLcademic ^ 

. areas, and ' ' , ' * x . . ^ , - . 

• ."Projects focusing on 'preparing parents to inpr^^^ 

the education ^d development of their chi'ldy^n 

(GAO,. 19.75, pPv,3-4) . . ; ^ ^ ^ / ' ' ^ 

ThiSvpap^r is 4esign6d- to explorar' the measured outcom§s of one of these 
educational strategics — /the behaviorally structured apf^roach. 
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GocflLs of the Behaviorally Structuj i'ed Mo'dels |i 

^ ^ — ' ' ' , . ' . »1 

Therp are two models which "caii be clearly ide^itified as 

' prescribing a behaviorally structured approach.. Drawing upon bebavioi^ist 
theories of dhildren's actionk as an interrela.ted sys€#m of stimuli and 
irespdn^es, these two models, haVe^ developed c{irricu£um approaches which 
focus on the assumption that Children's behavior ;is primarily i result of 
external stimuli. By carefuMy controlling the extei:nal stimul|L .these 
models attetoJJI^o modify the^ children 's" beha.vior in- ways which meet the 
specific goals '5^:^ .the" program. Meet^jig goals- is accomplished -by, clearly 
defining a series of discrete steps, and working toward' success at, each 
step. Praise and , rewards are provided for successful completion of each 

• step. 

• There are some specific di^erenUs between these two programs. 
First, let's label the models C^and a (tl -distinguish them from dther 
mo^^s discussed in the symposium) and^'tMen examine those differences. 
Model C's goals focus, on accelerating thgl learninghpf 'reading, math, and 
languaigfe skills tHrqilgh the use of programmed 'ihs^fucfcton . (Non-core 
subjects are generally introduced after mastery'of basic skills) usually 
toward tl}e end of . third gr,ade, ) The model T's , operatibnalized through 
the use of sponsorrdeveloped- programmed instructional materials in small 
homogeneous groups"^ in sbun^i-controlled booths. The .teachers- present a 
fast-movijig .series o,f -specific/question^ to elicit verbal student 
responses.. Propar' Student responses ai*e reinforced ajjd' wrong answers 
are corrected^ a9c^^:diiig tq speci^e^d procedures, 'student groups rotate 
by schedui)^^G;:ou]^ instruction 4.s ^ accompanied by individual, self- 
directed pi^^ctice ?n workbooks/ 'Pl^ned\homework' assignments are also * 
coordinated with .the glassr.oom lessoh^. Criterion reference tests ' ' 
administered to students a,t frequent ah'd regular intervals provide 
3^formatLon-^to the teachers on the- students^' progress, i ' 

- ^ Parents participate iil the program in several capacities • Some 
are employed in each classrdom on a permanent basi^' as* teacher aides 
(twp per fclassroom) and assistants/ O^th^rs are einpioyed as needed and 
trained to aaminister the criterl^bn-referenced pupil prpg^ress^ tests and 
operate the videotape egiiipment to f ilm the^ teacher at wojrk in the class- 
roon\. Still others are employedfks family vrorkers. . In this capacity, 




they acquaint parents with the Model C program, provide specially 

I 

develop'ed materials which parents can use at home to supplement classroom 
instiructioHr make available to those parents who so desire a sponsorr 
developed programmed course in child jnanagement, encourage particif)ation 
in PAC meetings, and assist in training the classroom aides and assistants. 
In sum, this model is designed to help children achieve quantJ^iable 
objectives in the basic skills ofljseading , arithmetic, and language ^ 
through the application of bohaviori^^p principles to instructional 
methodologies^ and materials,/ . ' * ^, 

Like Model C, Model D is highly structuj^ed, derived from behavioral 

. Jit 

psychology, and designed to increase students' reading arithmetic 
skills. In addition. Model D includes handwriting a^d spelling skilj 




/ 

as core - svibjects, and emphasizes the development of social skill^s^ as well 
Model D is based on the premise that the conscious and ayatem^ac: 
use of praise cuid af>proval will promote academic and^Hehavioral growth 
in the indivi<|tjal child.- The model is 'op^fationalized by^ establishing 
a ".token economy" within each classroo'ia. ^ "feachers award tokens for 
approv4||l social and academic peijf ormance * The^ children can- use these 
tokens during an exchange period tq purchase activities o/ their choice, 
such as games, toys/, books ^ etc.*^ ^tokens and praise cire distribu«d 
according to individual rates of progress rather than for group 
perfbrmance'. ^Teachers in this ^ model may choos,e among feitlier sponsorr- 
developed'or comipercial learning materials , .but they ape encouraged to 
select thdse^ v^icfih can be adapted to the model. Using a machine-- readable • ' 
data f oi:m, te6?chers prepare continuous progress ^reports on students.^ The ^ 
, da^a is then computer analyzed and an individual. 'ptogress prescription' 
is returned Vi thin, a day. \ ' ^ ^ ~ , 

Parents aiso play an* important role in the Model D classrooms « 
Two parents are, trairied and placed in each classroom for cux eight-week • 
period to. teach Handwriting and spelling, to- do individual tut^rin^', and 
to befcome knowledgeable^ in the use of the Model D techniques. At the 

. . ^ . , * ^ • ^ K ] . . * • 

6nd of th^ eighty weeks, the teaching pa^rents may con^tinue or not; as they ' 
choose^ • Although many parents s^'erve only " for an eigi\t-week session and 
fc^ach only *in on^ curriculum' area, some teach a full year-in as many as 
'thrfee curriculum areas. Many eventually become pemanent teachers* , aides. , 




In sum, acquisition of^o^i^cademic and.s<^ial skills are encouraged 
tKrough the systetnatixuse of positive reinforcement in the form of 
j>raise and'^fedlcens . - 



Imp lementation 

Determination of wh^^^^ op^ot a particular planhed vajriation 



ilpihent| (jf the leai 



Teache:^ Reports as Indicators ;3rf ^og^afa^ 

strategy is effective r^dires^oth k clear description of the essential 
elements' of the Mucat^na>^rogram ^ind some indication if whether pr 
not imp]lementa^ibn ij^s/ isfeen attained , In the preceding auction of this 
i>a^er 't^e ftoals^fpf bc|||i models arfe p resented with Ln expjlnation of the 
^i^^^^^L ^ ^t._ , / ig! environmenjtr' bofc^ withinithe ciassroom and at 

implementatijin, ho\feve5t, is not I 



there arel na^merous condition^ whicii \ 



IhomeJ |[4entificatiori|'of evidence of 
, / - ? . it' 

f readily available. Recognizing th^t 

could influence /the implementation fo|f the r^iodels, it still seems reasonable 

that after a minimym of five years operation there should be some I 

indications of implementation of the intended classroom environment whi|ii 

could be Identified from the data in the teacher questionnaire. The I 

teacher questionnaire is the only measure which provides us with data 

across all sites included in the analytic san^le and does t^flect -the 

perceptions of teachers who operatiqnalize the models daily in the 

classroom. While we recognize that jthese are. data on reported rather 

,ti^in observed 



practices, we nevertheless feel the information is valuablk 



in o^^er to assess implementation, or some proxy for implement::^tion, 
[we have! selected specific items from the teacher questionnaire administered 
to all jpollow Through and nOn-Pollow Through teachers at the end of 
third grade. ^ j . j 

Hgures 7 and 8 display :^ave dimensions of the classroom learning 
environment which can serve as proxy measures of the model's intended 
goals; teacher centeredness; flexible classroom; individualization of 
instruction; socio-emotional versus academic orientation; and teacher 
satisfaction with the model. Referring specificall^vto. Figure."?, Model C 
has generally implemented a structureld classroom learning environment 
as intended; however, there are some notable site variations in the degree 
of classroom structure. Site 005 and 081 have clearly implemented highly 
structured classroom learning environments whereas Sites 069 eind 072 
are much less 'structured. All si^es show below average individualization 
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Figu/e 1 

Indications of Implementation From teacher Questionnaire Clusters ^ 

for Sponsor C ' 
Behaviorally Structured Approaqh 
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of i^istruction. Three sites, 005, 070/ and 081, are very 'high in the 

degree'^of academic emphasis. This appears to indicate that the "desired*** 

degree of acadexaic etnphasis has not been achieved in at ledst two sites, 

especially 072. Teachers in three sites, 005, 072, and 070, report a' 

low degree' of satisfaction with the sponsor. Iri summajry, it aopears 

iihat the Model C is fairly well implemented,' except 'for site 072, .which 

is deviant^ . ' ^ * . - 

Figure, 8 shows that, generally .speaking, Model, D has implemented 

a structured classroom learning environmen* intended. -Model D 3ites 

also »show a high degree of individualization of instruction which is 

consistent With the model. In this'ca^e, there is ^considerably more site 

variation ^in .the degree ofi structure implemented. Sites 087 and 0^* . 

.appear to have less s true tu:red- programs than other Model D si-^^. , Sites! 

087 and 089 are' also considerably lower than others in the* degree of 

individualization and academic emphasis. Site 091 ^appears to have 

adopted a more flesg^ble classi^oom than mi^t have been ^^qpected from, the 
'.ft *^ » ^ 

model *s description. With the exception of sites "091 and 089> Model D 

\ : ' * \ * > ' . * ' " 

teachers express | less than average satisfaction with the sponsor. 

In summary, both Models C and D appear to be implemented in 

most sites . ' - . - ' . ^ ' 

i : . » • ' 

! : • • . ■ ■ • 

S ample - . ' . . ' ' 

The analyses for Model C are based on data from fiv-e sites,' . 

jlncluding one Harge northeastern city emd foxir cities in the midwest. ' 

Tl;ie Follow Through, childrein in these 'S.ites ax« very* similar to* the 

average Follow Ijl^rough child with respect tt>' such variables as ''pre school^ 

afttendance |(89% lat^tended pireschool) , enterinig pretest |scores (33 points 

on WRAT) , child ethnicity (88% Black 'chi]idrenL,*' and family ihcorroe ' 

.{$520(>/yr . ) . The same holds tirue for/ 1±ie Mn-Folloy/ ^rouc^ children ' 

included in the analyses of these sities . |That is," 47% attended pr^^chool 

chil^en-rScored 33 points on WRAT^at entr^; 80% cife Black children; cmd 

the average family income is $5308 — all similar to average for NFT 

children^ 'In general ^while there is variability across sites, the njodel 

' " \ ^ , ' ^ " \ \ 

average for both FT and NFT groups is sxmiiar to the r^pective average 

for .all Follow Through sites. 



* Model D, sampled' in six sites, has a considetably lower average 
family income, that is, $3890 with a rakge between $2075 and $4906. The 
average for the Follow Through ciiildren in Model D*s sites is also about 
$1000' lower than the average for the non-PoHow Through children in tiiese 
sitjes* Other variables such ^s pretest scores, percent with. presc^Iiool , 
and percent of mothers' with high s<fhool Education are similar to the 
average for all Follow Through sites.'. While the Follow Through children 
in^Model D*s sit^s tend to^?^ve^ a fe lightly higher pretest score in / 
comparison to the noH-Follov^. IhroAgh children, about 20% mor^ of * these 
children , have ^i-eschool exp^ienc 



Interoretation or- Curricul^ Imoafctf on Child Outcomes r f — ^> 

• - ~ ' — ; , ' ^ f — j 1 ; — / ^ :J 

To explore' the ^estiOnl 'SUo .the models result in impacts eithef 
pj^escribed i^ their statd^ '^oalk and objectives drAr^flected irt .other 
"areas measured by the te^'t I^at-dpry?" let ^ us ise two\ con^arispns oS da^a,* 
One, a 'comparison of the^^' 4^^^?!^^ I^®^^^™^^^ thk Follow JT^rou^ 
chiidreri with-^ the perforin^ce' fex^iected in the absencip of treatment. 

An(J-twQ, the comoarispiTf o^.tfii Follow Through 9hildr^n with 4he national 

' f ' i ' \ I • - ' ' ' ^ ' 

aorms fltoup foi: the MetrobQliluan' Achievemenh Test. 



Using the difference /petwee 



weeil observjed ^md 



/ 

. / 

predicted/ scores for t 



'FqIIow TlLrough children '(fevGaLed by the pooled comparison analysis) , ' 
, S3,^es can oe categorS/zed in^three ways: -1) Jpllow, Thrtfu^ performed 
better than would be^ predicted- without the prbgram; 2) Follow Ttjrdugfi; 
p^rf oi:med. as well^s| j^ould/be predicted; 3)- Follow Through performed 
less well than woxilq be predicted, Ss Figures 9 and 10 show, these 
.categories are distifngjuished. in terms of standard deviation 'linits as ^ 
follows: 1) if Fol^Low^ Through performs at plus a quarter of a. standard 
deviation unit bett^'er than noh-Follow Through, then we may say EolloWj 
Through performed ^ett^r thcin would be predicted without treatment; 
2) 'if the number is negative and greater than .25, we say Follow Through 
jperforms less well, than would be predicted without the program* 

First, let us examine the area of basic skills, .as measured by 
the heading, >t;ath; and spelling outcomes* . Looking at Figures 9 and 10 
it can be seen that there is considerable variability both across the 
sil:es within each model and within each site across outcomes. Both 
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Figure 9' , * 

Sponsor Effects on Seven Third Grade 
Tests, Expresses! in Standard Deviation' , 
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models have two sites which appear to perform better than would be 
expected without treatment on MAT Total Reading, Model C has one site 
and Model D two sites that are performing less well than would-be 
expected in Peading. In Math, Models C and D also have one axid two 
sites, respectively, performing less well than expected. With the 
exception of one site in Model D, all of the sites included in the 
behaviorcilly structured analyses perform less well in Spelling than 
would be expected without treatment. 

In the cognitive-con'ceptual domain ^ the Raven's Coloured Progressive 
Matrices, a test of prc^l em-solving ability, only one site in Model C 
perfonns be-tter than expected.* Otherwise, three land five sites, 
respectively, perform les^ well, them would be expected without the 



progra 



\ 



In tha ''affective ^omain,, the outcomes 'seen in Models C and D 
diffet^ dipnsideraiiy . While both !^odels C *aiJd D show at least four 
sites Iwhfere th^ Follow [Through chilflren perform better than would be 
expected on the Goopersmitri test of Iselfresteem, only Model D shows 
any material and significant [effects! on the Intellectiial Achievement 
Responsibility Sccile (lARS), a measure of the extent to which children 
take responsibility for themselves ot ^ttribute their successes to the 
operation of internal or external forces. In th^ case of Model D, 
(Figure 12) site 006 performs better ^an expected on the lARS (+) , and 
site 087 performs less well than would be ejroected.. On the acceptance 
of responsibility for negative events, as noted by "lARS^(-), in four 
sites Model D*s Follow Through children performed better than would be 
expected. Only 6ne site, 089, performs less well tham would be 
expected. 

In summary, both Models C and D appeeu: to be achieving some 
C|f their intended goals, at least in Reading and Math. (Model C>^ 
Langijage goal is not include^ in this ancilysis.) However , Mpuel D's 
Spiling gocil does not appear to be met. While Model D'sygoal for 
increased social performance and behavioral growth is n<yc assessed 
directly, there are indications that the children are performing better 



than would be expected in these areas. 



When we change the criteria to the legal definition of educational' 
. disadvaiitage*, i.e., performance which is one or more yeara b^low g-rade 
* level, an additional perspective is added to bur analysis*.. Conv.ersion 
to MAT national norms ^hown in Fig^jre Ij. reveals, that *all sites in. ^ 
Model C ,are perfonhing above the legal criterion of educational' ' 
disadvantage in Math, Reading, and spelling at the end of third grade, 
, For Model D (Figure 12) we see that, with one minor exception (site 089 . 
math norm)^, all sites are also performing aho|/e the legal criterion. 
Since entry level irs not controlled, it is problematic whether thir^ ' • 
grade MAT ndrm performance is attributable to treatment. HoweverV while . 
the pattern of effects for Model C appears to correlate .with the soci'b- 
econoT4ic disadvantage of the site (as Figure .11 shows), this is not^.e ' 
case vtLth Sponsor D (Figure 12) . Here, most of the Follow Through s\i:es 
are ini the lowest incpme category (less tj^an $4,700) but the gra^e 
equivat^ents of the chil^dren ^re s.till at pr near* grade level, 

J To summarize, it appears that bot^ behaviorally structured ; ' 
approaches are successful in meeting* the i!r goals. This is demonstrated 
oy t*|e fact that Follow Through child3?en/p.n many sites perform better 
thanj expecred without the program in the b^^^-'c skills of Reading' and ' 
Matn, and m the socio-emctaonal domain measured by the Coopersmith. 
In addition the FT childr^' in Model D perform better than Model C^S ' 
children on the lARS (r) ih particular. This seems to indicate that the 
approach taken by Model D has generated within the children the recognition 
that -they can take responsibility for both positive and neg'ative events. 
Model C'.s ajid D's progthms appear to differ primarily ii^ the (^egree of 
individualization of instruction: They also differ in the type of 
children they serve, with Mode l.O children being considerably more 
disadvantaged. Since both models -appear to meet most of their goals, 
it appears that the behaviorally structured approach is generally 
successful with disa^lvantaged children. Beyond that, however, a high 
degree of individu^li^zation within a structured nodel may be particularly 
appropriate for severely disadvantaged children/ Hence we conclude from the 
outcomes measured that the behaviorally structured .approaches are 
generally approfiriate models for (j^mpen^atory education, and that while a 
group instructional approach may be appropriate for a moderately disad- 
vantaged group, individualization of instruction may be necessary for a 
severely disadvantaged group. 
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' OPEN ci^^SSROOM- CORRICULAR models in PROJEiW' FOtLOM THROUGH ' - ^ 

^ ' Nancy Am*es * ' ^ . * \ * 

^ ^ ^ ."^ Abt- Ass6ciates I^c. *• » ^ ' ' ' - 

^ > ' * 7 -SS^Wheel^'r Street * * ' \ - • \ 

• ' V ' * ' . .Cambridge^ Mass. 02138^ ' T • / • ^. , J ' ^ /• 

Overview of ' the Open •Classroom Models ' ' \ / • ' ^ ^ T > 

' ^ '*The Open Classroom models share a psychodynamic theory of le;aming; 

Their .primary emphasis, is* pn the developmeht of "healthy" individuals with 
."^positivef self-images aijid sound inteirpersonal relationshi'^s. The learn^jigo 
. environment , is- flexible and supportive ot interpersonal freedom and self-* 

' determihation. * ; - 1 ^ • . ' * % 

The. Ope^ Cla^sjpom models difffei: front the Behavioral Tnoi:|els in both 
philosophy and approacjji. ^While the Open models value academic skills, ; 
which tl:^y see as essential for the optimal development of the "whole*^ ; 
phild, '.these skillls ^^'^ seen as secondary in impdi;tanc6( to a "healthy*' 
sense of self and a pbsi^tive at^titude toward learning.! The lattper socjlo^j 
emotional goals are seen as prerequisites for the former, Furti^ermore ^ . thdl 
Open .Classroom models favor an unstruQturedl environment in which both 
te^^che^r and child have a great deal of ' perspnal freedom, and the curric^j^ 
follows the chilcjren's in|t:erests and explorations. This' con tjraSts sh^^ly 
with the. Behaviorist inodeis iri which teacheij aid stiident beha[vi([)rs; are pre- 

determined, specific objectives are delineatedi^ and instinictiion', i^sl cafe- 

1 ' ' ' 
fully sequenced, » . ' 



' ,It is .more difficult to distinguish between the^^OpeaiCij^asproom and 
the Cognitive Development models,. The philosophical difference between 
theseyapproaches is' not fundamental and several Follow Through models could 
vbe classified in either^ approach. Both approaches Value the development of 
problem-solving sjcills as well as sQ&io -emotional competencies • For the 
^N^en Classroom models, however, affective development. is perceived as the 
primary objective. For the Cognitive Develo^ment>modeli / the reverse is 
true^ / ^ ' ' . 

For the.Jnl^oses of this symposium, we have ,categoi^i2ed -tvfp. of the' 

■^^^ r ^ \ , " 

Follow Through models as exemplifying the Open Classroom curr^:t^lar ap- 
? proach. We have labeled them E.and F. Although both exemplify the Open 



[approactr to education, there are differences between *t^e models in theore'^ti- 

c^l .origin ^ ^pptoach ; to* educational innovation, and change, content area " 
anphasis^ and curriculum offerings. Th^re are also differen'ces in eadh 
'tapde.I^s approaqh tp structuring the learning environment. 

. ♦ M<:xiel^E,is based , on 50 years qf experience in providihg educatiain , ' 
programs fpr young children.. This rabdel^ which is delsi^ned to develof) (^on- 
^structive^ cc|nfident, productive and.-coE|ing humcui bejinas^ attempts to * 
develop the *'whoie" child. .A te.iche?: training (Prograhi primarily, the model, 
B^eks to ,reabii ;ifes''o.bject|,ves th qomgh, improving the qlaai^ity of teaching".. 



Peachers aire* 'trained 



Arrange the^ ^ociaf milieu ot the' cla,ssroora, and provide instruction in 

f ' 1 ; . '11 ^ ' ^ ' ' • 

accordance, With indiv^idual learning needs. Teachers are given insights into 
iiAw to" enhance the capacity to probe/ reason, cuid express feelings., A wide 
Variety of ma^^rial^ is available for this model, including language arts 
and social studies prpgrams. The social studies curritulum is designed to 
^nhan<i:e interperspnal skill|s and develop cultural awajrfeness. 

. Model F has its roots in the, British ' Infant Scjftool approach 

dation.v ' Tha primary objective of this modal is creating a flexible, 

S 1^ ' > ■ 1 • ' \ 

supportive epvironment in which^the child's ability to learn is fostered by 

the educational setting. Since the 'model does not identify 'a single way of 

teaching chi|.dren, a unique, set df objectives, or a c^}inmon set o^ materials, 

teaching behaviors, classroom environments, ' and materjials vary widely. . 

^ Interest areas are an important aspect of t][ie learning environment in which 

children aire'^free to explore, to choose, and to manipulate. 



';tous'i^^.^.di agio Stic tools, ta analyse child behavior. 



o edu- 
open, 



Sample . ; 

The sample children for^^a^l analyses re'ported her^ are Cphort II 
children, who h^ve been in the Follow Through program from the beginning 
of kindergarten through the end of third grade. Six sites are included in » 
the 6uialysis for Model E. Four of these sites are large ^or medium-sized 
citie^ in the Northeast, one is a smalls Nortfieastern community, and. one is 
a mediiiin-sissed Southern community. Across all sites, children in this mod^JL 
,cL^e slightly below the Follow Through average^ on kindergarten entry scores, 
although they come from families with incomes slightly above the FT average^ 
Th4y do not dif:fer^ significantly frbm^the Fbllow Tlirough group as a whole \' 
in terms of fire^^lii:M:>V^experience or ethnic stetu?.' . 



^ '38. ^ 



A'. 



v.- 



There is great variability among ModeJ. E^l sites hdwever. One. site 
is well abovfe average on family income and entering^ ki^^idergarten scores. 
Mothers in this site , are better educated than the ajl^era^e for mother's of 
Follow Through children, and there are fewer female heads of households, , 
One, site is ^jell below average on both family income aiidj entering achieve-* 
at. The ,fctur odier. sitis are Ipw on pretest scores but not on family in- 
Ethnically, ,the sites vary; from 100 percenp Whitet to lOj) percer^t 




Ipclud^ed in[ .the analysis of Model F are three sites. Of thesey. two 
are! large cities and the ^o^er , a medium-sized cifty,^ All are in the.No^th-- 



ea|t or Northc^ntral region of ; the United State^, Across all ^ites, cnild- 
rfei 'participating in this model^ were willj below the Follow ^Thrd^gh average 

per:cent 



in achievement upon entity to kiln4erg|arten',' despite the fact that 
attended preschool, 1 Family income Across/ ^ites is sli<Jhtly 
aveifage* One hyndredj percent 0f the 1 c^^il^ren are Black.' 



Model F' s^sites .vary somewhat on.tpoth entry achievement stitu 



'h;i4 

\te'i^t sdores, one is fair 
's:^tes ajire above averag;e, 'the latter below^ 



ly 



income, Two.silteis are dlqfee 



:s 'and 

e follow Through average pn pr^- 



elow average, ton^ median *fan\ily income,, the 1 'former 



TecLcher ' Reports as Indicators of ^ Program Implementation \ 

J The data' in Figured 13 and 14 illustratd'^e , extent to Vhich the 
teachers in each of the sites for these models characterized th^r class- , 

rooms ^s ; . / ^ ^ , » « ' * 

' I, Teacher centered versus 'child center^d;\, * ^ ^ j 

2, Flexibly structured versus inflexibly s.tructured; 

3, Indivj/dualized versus grtoup-oriented;' and . , ' * 

4, Academically versus sc^cio-emotionally oriented. , >^ 

They also display teacher's satidl^ction with the model. ^ - \ 
Model ET's sites can be described as child-centered and flexible dn 
nature although there is some variability among sites asjto degree. 
Five of the six sites in this model are above average on individualization 
of in^ruction, while one is below. Sites are mixed in terms of their 
academic^ versus "emotional emphasis. Teachers in two ^ites favor academic ' 
objectivei^i^ teachers in another two favor emptional development;' and 
teachers' in\he other two show little preference. Finally, 
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Figur,e 13 

lndi9&tion$ of Implementation From teacher Questionnaire Clusters.^, 

for Sponsor E 
Psychodynamic Approach 
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satisfaction with the model varies widely with, one site extremely satisfied 
relative to teachers in other Wtes , tw<?*Yelatively dissatisfied, and the 
oti^jk neither satisfied rjor dyssatisfted. 

Model F has two s^te| Which a^e mote chil^' centered, flexible' and / 
individualized than most othfer Folloy Through sites, according to teacherj 
Je^rts. The third is also /child centered and flexible, but reports ^an 
J ^ average' degree of individuajlization. Two of the three sites emphasi/e 
^ emotional rather than academic development; the third shows no clear 

preference. All three sites report above average satisfaction with the 
sponsor. 

Overall, then, one could say that teacher reports ^reflect the 
4nodels' educational philosophy and approach. Most variability is_^^^&^n in 
the area of goals, with some teachers in both models reporting that their 
primary emphasis was' on the development; of emotional growth and others re- 
porting tfiat their primary^ focus was on €he promotion of academic competen- 
cies. ' ^ .'''^^ 

This inconsistency in goal statements may reflect varying degress 
of implementation of the models, with some sitfs embracing the socio- 
emotional gc^als of the psychodynamic approach, and others preferring ;the~ 
traditional academic goals ^of the primary school program. Or it might re- ^ 
fleet the philosophy of th^ approach itself. In focusing on the "whole" 
child, developing basic skills, prpblem-solving abilities^ and personal- 
social skills are sken as intertwined. 




Interpretation of IjCurricular Impact on Child QutcOTies 

Considerih^ next child outcomes at the end of third grade/ we 
examine curricularf, effects from two sets of dat^: first the contrasts of 
the Follow Through! groups with the pooled non-Follow Through group, and . i ' 
second the contrasts of the FT groups with the national achiev em ei^ norms ; , 
As can be ^een in Figure 15, Model E*s effects vary by outccmevaiid 
by site. On the Ralyen's Progressive Matrices test/ two of the six sites ' 
perf ormed^.'Setter thkn would have been expected without treatment and oniy 



one perfojjsiB&^^low the expected level.^ In qontrajjtr'all of the sites- 



scored below t^^^^^je^ppcted le3^al on the MAT , Spelling outcome -measuife. 
Effects,J.n Reading and Math varied, wjj^^ sites performing -above, ifTelov, and 
nest ^pecte'd<fl^V€ls. Fipally,^os>^ive effect^ tfere fdund in one affective 
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Sponsor Effects oil Sey^n Third Grade 
Tests, Expre^ed In Standard Deviation \ 
Units of Raw Score, For Sponsor E 
Psychodynamic Approach \ 
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area. Four of the sites h^d a positive impact on self-esteem as mea'sured 
by the i^bopersmith Self -Esteem Inventory and the other two sites scored 
at expected levels, - On the internalization' of respo;isibility-€or success 
measure, however, children in one site performed lower than would have been 
expected without treatment, and the others at expected levels. On the 
internalization of responsibility for failure measure, two sites performed 
below expected levels. ' 

There is no clear pattern visible re lasting teacher reports and 
child outcomes for this model, - ' ' ^ 

Turning to Figure 16, one can see that children participating in 
' Model F performed at or belo^ what would have 'been- expected without treat- 
ment on. all outcome measures, 
■ -4 

To summarize the comparison of the Open or Psychodynamic curricular 

models with the pooled non-EJolloV Through group, we can make the following 

generalizations: ^ ^ ' 

• In the cognitive and' affective domai¥is, these models do 
^ not prbdu^ similar effects. One model produces fairly 

consistent positive effects on the Raven's Progressive 
Matrices Test '-and the Coopersmith Self -Esteem Inventory, 
The oth|^ consistently does not, 

\^ • As for the academic ^are^a,\t:here is variability between 
'models and among sifees.^ One model generally performs 
lower than expected in this domain and the other -per- 
forms above expected LevelSj'^fn a limited number o^j^ites. 

The results of ^the regression analysis pifoylde estimates of a 

model' s performance compared to' what would be expected without treatment. 

In contrast, grades equivalent scbres provide '"information on the status of 

the Follow Through sites relative to the national norm group aj the end 

of the children' s tenure in the Follow Through program, ^ Bf course,*^ the •« 

national norm group, is a far less disadvantaged group, follow Through 

students genially would .fee 'expected .to perform we^ below this grou^) 

without treatment. To the/^tent th«?tJFT children are at or near grade , 

level at th© end of third grad^. Follow Through might *be said to be^ having 

some impact, Al§o, if -FT appear.s to interrupt the strong relationship 

between income ^and ar^ievemejcitir ,kkkri f¥ may be said to be h|.vinig[ an effect, 

Kjr*Q^n be seen in Figure 17, Mdiel T h^s two sites, pne-half year or less 

be^bw grad^ level,, two' sites less than one j^ear jDelow gy'ade level, and 

two more -sites more, than one year below grade level* Thus in four o'f th% 
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Sponsor Effects on Seven third Grade 
Tests, Expressed in Standard Deviation 
Units oif Raw Score, For Sponsor F 
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Figure 17 

Grade Equivalents and Standardized Test Scores on MAT Subtests 
(Reading, Math, and SoetTing) with Income Levels for 
COHORT II Sites in Sponsor E 
_ Psychodynamic Approach 
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six sites children in this model would not be classified as legally/ dis- 
advcintaged. Since one of these siteg is also relatively poor in median 
iricome, Fo'llow Through^jna^be said to be having a particularly strong im- 
pact in this site. \ **/ 

In contrast, Figure 18 reveals that Model F Was two of its three* 
sites performing one or more years "I^bIow. grade level, \^he other site is 
more than one year below grade level on one of the outcome measures, less 
than one year below on the other two. Figure 18; also shows that there is 
a correspondence between grade level performance and site m^an incane. 

^"-acx^vidence in these data that Model F has disrupted the 
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Qualitative Assessment of Curricular Impact 

Comparisons with either a pooled non-Follow Through groub or a 
national norm group £u:e by nature sensitive to the differences between the 
Follow Through and comparison groups and the nature of the FT group itself. 
These and other factors often make it difficult to separate ti^tment ef- 
fects frogi^^experimental error. The power of our tests is ^ch that effects 
must be quite large^to be observeii. If we ignQr:& _for a m oment the size of 
the eff ee^^^^^^nd examine ojily the pattem'^or ordering of effects across 
domains r we find a somewhat s^Scing^j^ttern^Jsee ly *" , . 

The Open approiSfiTapp^^rs to hav$ its greatest^-effeits in the af- 
f ectX^^-^^Qgig ^W pa rticularly in the area' of self-esteem. It ha^ 




strongest^ effects in the problem-solving area." Its weakest -effects _are in. 



/ 

Spelling, ^ somewhat rote learning task. ^ ^ ^ 

As pointed out in the introduction of this paper, it is difficult 
to separate the Open or Psychodynamic approach from the Cognitive Discovery 
approach. Both value an open, manipulative environment in wM^h children 
are encouraged to learn how to learn. This philosophical ^closness is ^' 
clearly reflected in the results for this approach ~ the strongest treat- 
■-ment_kif,ect;s appear to be in the cognitive and affective areas. 



Figyre 18 

Grade Equivalents anrf^artdardized Test Scores on MAT Sblptfests 
UMI'eading, Math, and Spelling) with Income Levels for 
^ COHORT II Sites in Sponsor F 
Psychodyna'mic Approach 
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